Near real-time streaming analysis of big fusion data

Abstract Experiments on fusion plasmas produce high-dimensional data time series with ever increasing magnitude and velocity, but turn-around times for analysis of this have not kept up. For example, many tasks are often performed in a manual, ad-hoc manner some after an experiment. In article we introduce the DELTA framework that facilitates near real-time streaming big fast data. By measurement from experiments to high-performance compute center, allows computationally expensive be between plasma pulses. This describe modular expandable software architecture present performance benchmarks individual components as well example workflows. Focusing workflow where Electron cyclotron emission imaging (ECEi) measured at KSTAR NERSC's supercomputer routinely observe transfer rates about 4 Gigabit per second. At NERSC, demanding turbulence effectively utilizes multiple nodes graphical processing units executes under 5 minutes. We further discuss how uses modern database systems container orchestration services provide web-based visualization. case ECEi demonstrate visualizations can augmented outputs machine learning models. providing session leaders physics operators results higher order using live may make more informed decisions configure next shot.